obj: recalculate curr_allocated on underflow#37
Conversation
rebuild heap_curr_allocated from on-media state if the counter has underflowed. Signed-off-by: Liang Zhen <gnailzenh@gmail.com>
Signed-off-by: Jan Michalski <jan-marian.michalski@hpe.com>
janekmi
force-pushed
the
liang/pmdk/pmdk_fix_curr_allocated
branch
from
f1aa619 to
483b054
Compare
Signed-off-by: Jan Michalski <jan-marian.michalski@hpe.com>
Signed-off-by: Jan Michalski <jan-marian.michalski@hpe.com>
Signed-off-by: Jan Michalski <jan-marian.michalski@hpe.com>
Signed-off-by: Jan Michalski <jan-marian.michalski@hpe.com>
janekmi
changed the title
grom72
left a comment
grom72
left a comment
There was a problem hiding this comment.
@grom72 reviewed 10 files and all commit messages, and made 1 comment.
Reviewable status: all files reviewed, 1 unresolved discussion (waiting on gnailzenh, osalyk, and sherintg).
src/test/obj_ctl_stats_curr_allocated_wa/obj_ctl_stats_curr_allocated_wa.c line 38 at r1 (raw file):
ret = pmemobj_ctl_get(pop, "stats.heap.curr_allocated", &allocated); UT_ASSERTeq(ret, 0); UT_ASSERTne(allocated, 0);
We know expected value as we check it in log match.
Suggestion:
ret = pmemobj_ctl_get(pop, "stats.heap.curr_allocated", &allocated);
UT_ASSERTeq(ret, 0);
UT_ASSERTeq(allocated, 0);
PMEMoid oid;
ret = pmemobj_alloc(pop, &oid, 1, 0, NULL, NULL);
UT_ASSERTeq(ret, 0);
ret = pmemobj_ctl_get(pop, "stats.heap.curr_allocated", &allocated);
UT_ASSERTeq(ret, 0);
UT_ASSERTeq(allocated, 128);
Update PMDK to incorporate the following fixes: - fix "The pool was not closed" message (no ADR failure) daos-stack/pmdk#36 - recalculate curr_allocated on underflow daos-stack/pmdk#37 Signed-off-by: Tomasz Gromadzki <tomasz.gromadzki@hpe.com> Priority: 2 Allow-unstable-test: true Focus validation on PMem version Skip-func-hw-test-medium: false Skip-func-hw-test-medium-md-on-ssd: true Skip-func-hw-test-medium-vmd: false Skip-func-hw-test-medium-verbs-provider: false Skip-func-hw-test-medium-verbs-provider-md-on-ssd: true Skip-func-hw-test-large: false Skip-func-hw-test-large-md-on-ssd: true Signed-off-by: Tomasz Gromadzki <tomasz.gromadzki@hpe.com>
Update PMDK to incorporate the following fixes: - fix "The pool was not closed" message (no ADR failure) daos-stack/pmdk#36 - recalculate curr_allocated on underflow daos-stack/pmdk#37 Signed-off-by: Tomasz Gromadzki <tomasz.gromadzki@hpe.com> Priority: 2 Allow-unstable-test: true Focus validation on PMem version Skip-func-hw-test-medium: false Skip-func-hw-test-medium-md-on-ssd: true Skip-func-hw-test-medium-vmd: false Skip-func-hw-test-medium-verbs-provider: false Skip-func-hw-test-medium-verbs-provider-md-on-ssd: true Skip-func-hw-test-large: false Skip-func-hw-test-large-md-on-ssd: true Signed-off-by: Tomasz Gromadzki <tomasz.gromadzki@hpe.com>
... move recalculation to the getter. Signed-off-by: Jan Michalski <jan-marian.michalski@hpe.com> Co-authored-by: Tomasz Gromadzki <tomasz.gromadzki@hpe.com>
Signed-off-by: Jan Michalski <jan-marian.michalski@hpe.com>
Update PMDK to incorporate the following fixes: - fix "The pool was not closed" message (no ADR failure) daos-stack/pmdk#36 - recalculate curr_allocated on underflow daos-stack/pmdk#37 Signed-off-by: Tomasz Gromadzki <tomasz.gromadzki@hpe.com> Priority: 2 Allow-unstable-test: true Skip-func-hw-test-medium: false Skip-func-hw-test-large: false
Signed-off-by: Jan Michalski <jan-marian.michalski@hpe.com>
grom72
left a comment
grom72
left a comment
There was a problem hiding this comment.
@grom72 reviewed 5 files and all commit messages, and made 1 comment.
Reviewable status: all files reviewed, 2 unresolved discussions (waiting on gnailzenh, osalyk, and sherintg).
src/libpmemobj/stats.c line 64 at r2 (raw file):
*/ if (*argv > *pop->heap.sizep) { /* covers the == UINT64_MAX case */ /* if the value is broken, recalculate it */
CORE_LOG_WARNING is missing.
osalyk
left a comment
osalyk
left a comment
There was a problem hiding this comment.
@osalyk reviewed 9 files and all commit messages, and made 1 comment.
Reviewable status: all files reviewed, 2 unresolved discussions (waiting on gnailzenh, grom72, and sherintg).
src/libpmemobj/stats.c line 64 at r2 (raw file):
Previously, grom72 (Tomasz Gromadzki) wrote…
CORE_LOG_WARNING is missing.
👍
Signed-off-by: Jan Michalski <jan-marian.michalski@hpe.com>
Signed-off-by: Jan Michalski <jan-marian.michalski@hpe.com>
janekmi
left a comment
janekmi
left a comment
There was a problem hiding this comment.
@janekmi made 2 comments.
Reviewable status: 6 of 13 files reviewed, 2 unresolved discussions (waiting on grom72, osalyk, and sherintg).
src/libpmemobj/stats.c line 64 at r2 (raw file):
Previously, osalyk (Oksana Sałyk) wrote…
👍
Done.
src/test/obj_ctl_stats_curr_allocated_wa/obj_ctl_stats_curr_allocated_wa.c line 38 at r1 (raw file):
Previously, grom72 (Tomasz Gromadzki) wrote…
We know expected value as we check it in log match.
Please see the update version.
osalyk
left a comment
osalyk
left a comment
There was a problem hiding this comment.
@osalyk reviewed 7 files and all commit messages, and made 1 comment.
Reviewable status: all files reviewed, 2 unresolved discussions (waiting on grom72 and sherintg).
Signed-off-by: Oksana Salyk <oksana.salyk@hpe.com>
5 participants
heap_curr_allocatedalthough is stored persistently it is not updated transactionally nor reliably persisted. This means e.g. that in case a transaction succeeds but the process will get terminated before the update ofheap_curr_allocated, the value ofheap_curr_allocatedwill get out of sync with the actual heap state.The most obvious case of this happening is when
heap_curr_allocatedis actually smaller than the sum of the sizes of all allocations in the heap, so in case all of the allocations are freed,heap_curr_allocatedwill underflow and get a very big value, bigger than heap size.Ref: https://daosio.atlassian.net/browse/DAOS-18882
This workaround detects this most obvious case and recalculates.
This change is